Image processing occupancy sensor

ABSTRACT

A system and method of detecting occupants in a building automation system environment using image based occupancy detection and position determinations. In one example, the system includes an image processing occupancy sensor that detects the number and position of occupants within a space that has controllable building elements such as lighting and ventilation diffusers. Based on the position and location of the occupants, the system can finely control the elements to optimize conditions for the occupants, optimize energy usage, among other advantages.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a non-provisional application claimingpriority under 35 U.S.C. §119(e) to provisional application No.61/261,667 utility filed Nov. 16, 2009, which is hereby incorporated inits entirety by reference herein.

The United States Government has rights in this invention under ContractNo. DE-AC36-08GO28308 between the United States Department of Energy andthe Alliance for Sustainable Energy, LLC, the Manager and Operator ofthe National Renewable Energy Laboratory.

FIELD

The present disclosure generally relates to image processing occupancysensors and building management systems and methods.

BACKGROUND

Building automation systems are a common feature in many commercialbuildings and are becoming more common in residential buildings.Generally speaking, a building automation system or “BAS” includescomputer hardware and software that monitors and controls variouspossible mechanical and electrical systems in a building. For example,the BAS may involve a computer system in communication with heating andlighting control systems. Occupancy sensing through either ultrasonic orinfrared based sensors is a common feature of many BAS. With occupancysensing, a BAS is able to provide lighting control and climate controltaking into account the presence or absence of occupants.

Conventional BAS systems using ultrasonic or infrared occupancy sensing,however, suffer from some drawbacks. Conventional occupancy sensingsystems often suffer from false positives—falsely identifying anoccupant in a space or falsely indicating a space is empty. For example,conventional occupant systems can lose track of an occupant in a spacewhen an occupant is within a space but is still, such as when someone issitting still at their desk. Similarly, a blowing curtain can falsely bedetected as the presence of an occupant in a space. False positivesoften result in inefficient building control and can cause occupants totemporarily or permanently disable the sensor.

Sensor manufacturers and BAS engineers have ameliorated some of theshortcomings in conventional occupancy sensing systems throughsensitivity adjustments, e.g., more or less motion over more or less aperiod of time required to trigger a sensor, and motion time-outs, e.g.,if no motion is detected for a period of time then assume no occupant,as well as complex advanced signal processing, fuzzy logic, andprobabilistic models. While some of these improvements to conventionaloccupant sensors may prove beneficial, shortcomings in such systemspersist.

The foregoing examples of the related art and limitations relatedtherewith are intended to be illustrative and not exclusive. Otherlimitations of the related art will become apparent to those of skill inthe art upon a reading of the specification and a study of the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments are illustrated in referenced figures of thedrawings. It is intended that the embodiments and figures disclosedherein are to be considered illustrative rather than limiting.

FIG. 1 is a system diagram illustrating an image processing occupancysensor in a building automation system environment;

FIG. 2 is a flowchart illustrating a method of occupant detection andusing the detection to control building elements;

FIG. 3 is a flowchart illustrating a method of occupant detection andposition identification to control building element;

FIG. 4 is a top view of graphical user interface for a user to define aspace including controllable elements in the building systemenvironment, the space subdivided into regions;

FIG. 5 is a zone map of the space of FIG. 4, the zone map defining cellswithin each region;

FIG. 6A is an illustration of using a detected image to determine theposition of the occupant within the space of FIG. 4

FIG. 6B is an illustration of Canny edge processed image and a baseimage used to determine the presence and position of an occupant withinthe space of FIG. 4;

FIG. 7 is a zone map of FIG. 5 including occupant data associated with aparticular cell;

FIG. 7A is a smoothing function;

FIG. 8 illustrates the zone map of FIG. 7 processed according to thesmoothing function shown in FIG. 7A;

FIG. 9 illustrates the various smoothed data of FIG. 8 offset;

FIG. 10 illustrates the smoothed and offset data of FIG. 9 normalized;and

FIG. 11 illustrates the zone map with the mean of the normalized dataapplied on a region-by-region basis, the region-by-region mean valuesproviding, directly or indirectly, control attributes for the buildingautomation system to control various possible controllable elements,such as dimmable lighting ballasts, ventilation diffusers, heating,cooling, and daylighting applications associated with the space, eachregion, or each cell.

Corresponding reference characters and labels indicate correspondingelements among the view of the drawings. The headings used in thefigures should not be interpreted to limit the scope of the claims.

SUMMARY

The following embodiments and aspects thereof are described andillustrated in conjunction with systems, tools and methods which aremeant to be exemplary and illustrative, not limiting in scope. Invarious embodiments, one or more of the above-described problems havebeen reduced or eliminated, while other embodiments are directed toother improvements.

One aspect of the present disclosure involves a method of controllingbuilding equipment that comprises receiving an image from an imagesensor placed in a space associated with the building. The imageincludes information representative of one or more occupants in thespace. At a first processor coupled with a memory storing the image, themethod involves determining, from the image, a presence of one or moreoccupants within the space. Based on the presence of occupants in thespace, the method further involves providing a control signal to one ormore controllable building elements, e.g., lighting or ventilation,based on the presence of the occupant within the space. The method mayfurther involve determining a position of the one or more occupants inthe space. Further, from a second processor in communication with thefirst processor, such as a BAS, providing the control signal to the oneor more controllable building elements based on the position of theoccupant in the space.

Another aspect of the present disclosure involves a building managementsystem comprising an image processing occupancy sensor. The imageprocessing occupancy sensor includes at least one image sensor and atleast one first microprocessor coupled with at least one memory. Themicroprocessor and memory are configured to receive and store an imagefrom the image sensor, the at least one memory including computerexecutable instructions including a first set of instructions toidentify an occupant within the image and a second set of instructionsto determine the position of an occupant in a space associated with theimage processing occupancy sensor and provide data indicative of theposition of the occupant in the space.

In addition to the exemplary aspects and embodiments described above,further aspects and embodiments will become apparent by reference to thedrawings and by study of the following descriptions.

DETAILED DESCRIPTION

A need exists for systems and methods that detect occupancy using imageprocessing. Such systems and methods can control, at different levels ofspecificity, elements in a space such as lighting and ventilationdiffusers, based on the presence of an occupant as detected by the imageprocessing occupancy sensor and related components. Such systems andmethods can further use the image data to not only detect the presenceof one or more occupants in the space but also determine the occupants'position within the space. Based on position, fine grained control ofvarious elements is possible. For example, lighting control in a spacemay be based on the position of occupants in the space, with lightingelements nearer to the occupant brighter than lighting elements furtherfrom the occupant. Moreover, other inputs, such as those related totemperature, daylighting conditions, CO2, user activity and the like maybe used in conjunction with image based occupancy detection data toprovide element control. For example, daylighting may be detected by theimage processing system or otherwise provided, and used in conjunctionwith the lighting control to decrease lighting based on occupancy dataand daylighting data (during bright days) or increase lighting based onoccupancy and daylighting data (during darker days).

FIG. 1 is a system diagram illustrating a building automation system(BAS) 10 and related components, including an image processing basedoccupancy sensor 12, deployed in a building 14. FIG. 2 is a flowchartdescribing one possible method conforming to aspects of the presentdisclosure. Referencing FIGS. 1 and 2 and more particularly, the imageprocessing occupancy sensor (IPOS) is deployed in a room 16 of thebuilding 14. The IPOS 12 may include any number of different analog ordigital image capture devices including a charge coupled device (CCD)and a complimentary metal-oxide-semiconductor (CMOS) active pixelsensor. The IPOS is positioned within the room 16 or other space of thebuilding such that a field of view of the sensor encompasses portions ofthe space where a controllable element 18, such as ventilation orlighting, is provided and under control of the BAS. In various possibleimplementations, one or more IPOS may be positioned within a spacehaving controllable elements, and the IPOS may be indoors or outdoors.Examples of controllable elements include fans, dampers, lightingfixtures, air handling units, heating and cooling systems and hot watersystems. The IPOS device is configured to detect the presence of one ormore persons within the room or other space (operation 200).

The IPOS device, includes, is coupled, or otherwise is in communicationwith an image processing system 20. The image processing system mayinclude a processor in communication with memory, whether on board theprocessor or connected thereto, and various other electronic componentsto interact with the image sensor and to communicate with the BAS. TheIPOS device may be positioned on the same printed circuit board orotherwise proximate to the image processing system or the imageprocessing system may be remote and physically separate from the IPOS.In one implementation, the image sensor is deployed in a stand-aloneframe structure that may be mounted in a desired building location. Theimage sensor may communicate with the image processing system by way ofwired or wireless Ethernet, Bluetooth, Zigbee, MSTP or othercommunication arrangements. In such an implementation, the term IPOSencompasses both the image sensor and the image processing system.

While FIG. 1 illustrates a single image sensor 13 and image processingsystem 20 in communication with the BAS, implementations of the systemsdescribed herein may include any number of IPOS devices. The IPOSdevices may include dedicated image processing systems. It is alsopossible to couple a plurality of sensors to an image processing systemsubject to the processing and memory constraints of the image processingsystem as well as other factors. In one particular arrangement, theimage processing system includes an embedded microcontroller thatprocesses the image data from the image sensor to extract occupant andother information. The image processing system, in this particularimplementation, communicates with the BAS over a conventional protocolsuch as BACnet over wired or wireless transport layers. Irrespective ofthe deployment of IPOS devices, a given image processing systemcommunicates with the BAS and provides image information gathered fromthe IPOS to the BAS. The IPOS processes the image data to extractinformation about the occupation of the space (operation 210). Forexample, the IPOS may determine the number of occupants in the space,the location of the occupants in the space, the centroid occupancy,space luminance, and/or other information and pass that information tothe BAS.

Based on the processed image data and occupant information derivedtherefrom, the BAS is able to optimize building control (operation 220).For example, the BAS generates a control signal delivered to a buildingsystem actuator 22 that produces the appropriate outputs to operationventilation systems in a space. It is also possible to configure theimage processing system 20 or some other processing element orcombination of processing elements, which may be in any form ofcommunication (e.g., wired network connection, direct connection,wireless connection) with the sensor device, to process the image datato determine occupant position, movement, centroid of occupancy,luminance and other occupant and space information based on the imagecapture.

To illustrate aspects of the disclosure, in one possible implementation,a large room is configured with a number of individually dimmablelighting elements, such as a dimmable ballast. A dimmable lightingelement typically includes some form of lighting element, such as anincandescent bulb, a light emitting diode light, or a fluorescent bulb,that provides a controllable amount of voltage and current to thelighting element so that the lighting element may be controlled (dimmeror brighter). The BAS system, in response to processing the image data,provides control signals to the lighting elements to individuallycontrol the light provided from each lighting element. Hence, asdiscussed herein, lighting within a space can be tailored based onoccupancy as well as other factors.

FIG. 3 is a flowchart illustrating one particular method conforming toaspects of the present disclosure. The various operations recited inFIG. 3 are discussed herein with reference to the diagrams set forth inFIGS. 4-10. Referring now to these figures as well as earlier discussedFIGS. 1 and 2, prior to capturing and processing image data or any formof resulting building control, a space under control of a BAS system andincluding an IPOS device is associated with a zone map or matrix for thespace (operation 300). In one particular implementation, a zone map fora particular space is generated by way of user interaction with anapplication accessible at a monitor, a personal computer or otherinterface 24 connected to the BAS 10. The user accesses a zone maptemplate by way of a graphical user interface (GUI) where the user isable to define a space having an IPOS and may further define regions ofthe space and cells within each space.

FIG. 4 illustrates a space 26 that includes an IPOS device 12 and FIG. 5illustrates a matrix 28 associated with the space. The matrix isprovided as a programmable or definable template at the GUI; initially,the user establishes the overall dimensions of the space and arrangesthe matrix in accordance with the dimensions and geometry of the space.The overall space is presented as a matrix with a plurality of cells 30.The cells may be clustered or otherwise associated with regions 32 ofthe space and matrix. Each cell, in one implementation, may beassociated with a controllable element either within the cell orotherwise associated with the cell or the region that includes the cell.For purposes of example, FIG. 4 illustrates dimmable lighting elements34 in one-to-one correspondence with cells 30. Other forms ofcontrollable elements are possible and it is not necessary that there beone-to-one correspondence. Generally speaking, regions will includevarious controllable elements in some form of arrangement and cells willbe defined, perhaps automatically once the space geometry and IPOSlocation is defined, in a distribution associated with occupancy and notnecessarily in alignment with controllable element locations. In FIG. 5,each cell is associated with a respective region. More specifically,region 1 includes 9 cells (each labeled 1), region 2 includes 12 cells(each labeled 2), region 3 includes 9 cells (each labeled 3), and region4 includes 6 cells (each labeled 4). There is a region of the space thatis not open or otherwise visible to the IPOS (the space could beassociated with a wall for example) and the cells in that region arelabeled −1. Hence, the overall space footprint is defined by regions1-4.

In operation, the IPOS detects the presence of an occupant or occupantswithin the space (operation 310). The system may also detect the numberof occupants. It is also possible that the IPOS will be configured todetect other characteristics of the room, such as thermal changes (ifthe IPOS has infrared spectral characteristics or is functionallyconnected to device that can detect thermal changes and ambientlighting) or luminance either alone or in conjunction with detecting thepresence of an occupant within the room. The IPOS 12 is positionedwithin the space 26 such that a field of view of the IPOS deviceencompasses an area of the associated matrix. The image data gatheredform the IPOS device is temporarily saved in the memory of the imageprocessing system 20. Image data may be gathered and saved at varioustimes and in response to various triggers. For example, the image datamay be captured and loaded into memory at regular intervals,intermittently captured and loaded into memory upon detection ofmovement, in response to detection of a change in light (e.g., in adaylighting application) and some combination of regularly and inresponse to triggers from the space, or otherwise.

In order to detect an occupant in the space, the image data is analyzed.One advantage of using image data to determine occupancy is that theoccupant may be detected regardless of movement. Hence, a person sittingstill and reading at their desk will be detected whereas someconventional systems lose track of the person sitting still and issuecontrol signals, such as turning off lights, based on the erroneousdetection. The analysis may be performed by the BAS 10, the imageprocessing system 20, or any processor configured to receive an imagefrom the IPOS. Occupant detection may be performed through Canny edgedetection, blob detection or other image processing techniques. WithCanny edge detection, the IPOS captured image data is processed togenerate an image highlighting the edges in the original image. In animage of a space that includes an occupant, the Canny edge detectionprocessed original image results in an image showing the outline of theperson or persons in the space. The Canny processed image is thencompared to typical human profiles to identify a match and hence thepresence of one more people within the room. It is possible that severalimages from the IPOS device may be captured and repeatedly compared tothe human profile information. Such successive or repeated comparisonsmay be compared to human profile

Upon capturing an image of the space and detecting an occupant withinthe space, the system identifies the location or locations of theoccupant or occupants within the room (operation 320). In the firstoperation, while defining the zone matrix for the space, the matrix isassociated with dimensions of the space as well as the location of theIPOS. Further, the IPOS device may be registered against various staticfeatures in the room, such as corners of the room or pillars in theroom, and dimensional associations are established between the IPOSdevice and the static features. FIGS. 6A and 6B illustrate one exampleof occupant position identification with respect to the space 26 of FIG.4, each corner of the room is registered in the IPOS (R1, R2, R3). TheIPOS includes one or more reference occupant images, which may be in asimilar image outline form as generated through Canny edge detection.The reference occupant images may include a set of reference images foran average size adult. Referring to FIG. 6B relative to theregistrations (R1, R2) the image shown places the occupant 36 in thespace 26. The processed image 36 is then compared with the referenceimage 38 (dashed) or images to determine the dimensional differencebetween the captured image and the reference image. The difference isthen used to determine the distance (D1; FIG. 6A) of the occupant fromthe IPOS device. The position (D2, D3) of the occupant 36 is based onthe relative positioning of the occupant 36 compared to registrationsR1, R2 and R3. The IPOS, knowing the space dimensions and registrationfeatures, which can be captured in the image data, is thus able todetermine the position of the occupant(s) in the space.

It is also possible to configure a given space with two or more IPOSdevices and use a form of triangulation to identify the position ofoccupants within the space. For example, if each IPOS device isinitially defined, through the GUI, at a fixed position in the space andthe registrations and relative distance between the IPOS devices areknown, one of the IPOS devices, the BAS system, or other computingsystem can use triangulation to determine the location of the occupantin the space.

When the positions of the occupants in the space are identified, thepositions are assigned to a cell in the zone matrix for the space(operation 330). For example, as shown in FIG. 7, the number 4 isassigned to the center left cell of the region 1 portion of the zone map28. This means that the system determined that four occupants werewithin the space 26 and generally located in the area associated withregion 1 and particularly the area of region 1 associated with theidentified cell.

Turning now to FIG. 8, the occupancy data (at the cell level) isprocessed according to a smoothing function that distributes theoccupancy across cells adjacent the cell including the occupant(s)(operation 340). Smoothing the occupant data establishes a point fromwhich the BAS may optimize control of the occupied space and itssurroundings. With the initial occupant data, the BAS knows whereoccupants are positioned in the space, within some margin of error inthe system. It is possible to initiate control of various elements basedon occupant presence or position data without any smoothing. Forexample, all room lighting may be switched on when an occupant isdetected within the room. Or, a lighting element associated with theoccupant cell may be turned on, alone or with adjacent lightingelements. Smoothing the occupant position data, however, allows thesystem to more precisely fine tune or optimize element control andenhance performance robustness. For example, as discussed below, aftersmoothing, lighting elements 34 associated with the area of occupancymay illuminated at varying degrees of brightness which enables optimallighting for the occupant in the space and optimal energy usage byreducing illumination at points further from the area of occupation.

In one example, a smoothing function, represented by FIG. 7A, is used tosmooth the occupancy data. The width and steepness of the smoothingfunction may be user programmable and hence the smoothing of theoccupancy data may be programmable. It is also possible to construct anappropriate smoothing function using a pair of interacting sigmoidfunctions, a sinusoidal function, a Gaussian distribution, etc. toproduce a gradually diminishing value centered around a position. Onepossible result of application of the smoothing function, using theoccupancy data cell assignment from FIG. 7, is shown in FIG. 8. Here,cells immediately adjacent (either directly above, below, right or left)the occupant cell 42, are assigned a value of 3.2. Cells angularlyadjacent the occupant cell are assigned a value of 3. Other cells,depending on their distance from the occupant cell, are assigned valuesof 1.9, 0.9, 0.5, and 0, while the cells associated with the space notvisible to the IPOS device remain −1.

In the particular methodology depicted in FIG. 3, the smoothed occupancydata is shifted (operation 350). More particularly, the smoothedoccupancy cell data values are each offset by an integer value of 1 asshown in FIG. 9. Shifting the occupancy data is used as a mechanism tofurther process the data in later steps. Like other operations set outin FIG. 3, operation 350 may not be necessary depending on the givenimplementation. Moreover, it is possible to shift the values by someother value, whether integer or otherwise, depending on theimplementation. Shifted by 1, the -1 values become 0, the 0 valuesbecome 1, the 0.5 values become 1.5, etc.

After shifting (or offsetting) the matrix cell values, the cell valuesare normalized (operation 360). In the example illustrated in FIG. 10,the shifted cell values are normalized in a range of 0 to 1. Hence, thelowest shifted value of 0 is normalized to 0, and the highest shiftedvalue of 5 is normalized to 1. The remaining values are distributedacross the range of 0 to 1. In the example of FIG. 10, a normalizationmultiplier of 0.2 is used (5×0.2=1) and values are rounded up to thenearest decimal value. For example, the cells with a value of 4.2 arenormalized to 0.9 (4.2×0.2=0.84, rounded up to 0.9). In other examples,cell values of 4 are normalized to 0.8 (4×0.2=0.8, no roundingnecessary) and cell values of 2.9 are normalized to 0.6 (2.9×0.2=0.58,rounded up to 0.6).

Control optimization may occur based on the normalized cell values(operation 380). For example, lighting control signals may betransmitted to one or more lighting elements associated with one or morecells where normalized values function as a dimming multiplier. Again,for illustrative purposes a one-to-one correspondence between elementsand cells is illustrated, but no such one-to-one correspondence isnecessary. In the example shown, a cell value of 1 results in fulllighting, a value of 0.9 delivers 90% brightness (10% dimming), and soon. However, as shown for example in FIG. 11, in the particularimplementation illustrated in FIG. 3, following normalization, thenormalized cell values are assessed on a region by region basis toobtain a mean value for each region (the sum of the cell values for agiven region divided by the number of cells in the region) (operation370). Hence, for example, relative to region 4, the mean value is(1+0.9+0.9+0.9+0.8+0.8+0.6+0.4+0.4)/9=0.75.

The region based multipliers are then used to produce signals to variouspossible controllable elements of the space in order to optimize controlfor the overall space. Continuing with the example of illumination, themultipliers may be applied to control illuminations to various lightingelements within each region (34(1)-34(4)). In such an example, anadditional normalization step or other factoring may occur such that thehighest value is associated with 100% illumination. The lighting elementcontrol signals are then applied on a regional basis such that all ofthe elements within region 1, associated with the highest regionmultiplier, are provided with a 100% illumination signal (no dimming).

The multipliers may be used in conjunction with a daylighting provisionsuch that the multipliers are normalized to a value less than 1 (lessthan 100% illumination) depending on other daylighting information (timeof day, date, weather, etc.). The term “daylighting” refers generally tothe use of windows, reflective surfaces, and the like to provide naturalindoor lighting, and “daylighting” information refers to informationthat the BAS uses in order to determine the amount of natural indoorlight in any particular environment. For example, the BAS system may beprogrammed with daylighting measurements for a given region taken atvarious times and across various days of the year. The system may alsobe programmed with a nominal daylight value (such as through one or moremeasurements) and the nominal daylight measurement may be adjusteddepending on the time and day of the year. The system may also beprogrammed to capture daylighting values for a given region. Forexample, the IPOS device may be programmed to capture an image of thespace whenever it is unoccupied and not artificially illuminated, andthe BAS system may compute one or more luminance values for the space,each region, or otherwise, and use the luminance value (or values) asthe assumed daylight value for the area. The daylight values may then beused in conjunction with the region multipliers to optimize illuminationfor the space. The IPOS device may also capture values over time todevelop a daylighting database of luminance values, with each valueassociated with a day and time, and use the database values inconjunction with the region multipliers to provide illumination control.Hence, to account for daylighting in an occupied region, such as region1, an illumination value of less than 100% may be sent to the elementsin region 1. The overall illumination of a given space will be optimizedto provide the occupants with sufficient light while taking advantage ofnatural light such that less energy may be used to artificiallyilluminate the space.

In order to account for occupant movement in the space and possibleerrors in assessing occupant position, it is possible that images willbe captured and processed at regular and relatively fast intervals,e.g., 60 times per minute. As occupants are detected in different areasof the space or move through the space, illumination and other forms ofcontrol may have to change. In the specific example of illumination,abrupt and quick changes may be distracting to the occupants; hence, itis possible to process the regional multipliers with an asymmetric-timedomain filter or other form of filter to account for and otherwiseattenuate quick changes. In the specific case of an asymmetric-timedomain filter, the filter may have fast rising and slow fallingcharacteristics. Hence, for example, as an occupant moved upward fromregion 1 to region 2, region 1 values may decrease relatively slowlycompared to region 2 values that may increase quickly. Hence, theillumination of region 2 may increase quickly as the occupant movedtoward and into region 2, whereas illumination of region 1 may decreaseslowly as the occupant moved from region 1 into region 2.

In another example, the regional multipliers generated in operation 370are applied to nominal ventilation commands for direct ventilationcontrol in each region. For example, ventilation ducts in a given spacemay be fitted with diffusers to control the amount of air flow form theventilation ducts into the space. In such an implementation, a nominalventilation command to region 2 ventilators (e.g., diffusers) may beprocessed according to region 2 multiplier of 0.26. If a signal for fullopen diffusers is provided to the region 2 diffusers, the signal may bereduced to 0.26—the diffusers being open to 26% of maximum in region 2rather than full open. Hence, air is provided to region 2 but less airthan under conventional operation conditions (i.e., without image basedoccupant detection and position sensing).

Aspects of the present disclosure may also be used to determine acentroid of occupancy in a region or entire space, and elements withinthe region or space controlled accordingly. To illustrate the use ofcentroid, take for example the case where the system detects singleoccupants in the center of each of the four regions depicted in FIG. 4.The system will calculate a centroid of occupancy for the space based onthe individual occupants in each region. The centroid may be located atapproximately the intersection of the four regions near the middle ofthe space. In this example, the centroid indicated as a single occupant,the four occupants, or otherwise, may be assigned to the cell associatedwith the calculated position of the centroid. This value may be used inoperation 330 of FIG. 3, and the subsequent steps may be performed togenerated control values in order to control various elements based onthe centroid of occupancy. In such a configuration, the smoothingfunction may be more gradual to spread illumination and ventilation moreevenly across the room from the centroid. Additionally, the smoothingfunction may take into account the number of occupants in the space aswell as other factors.

The description above includes example systems, methods, techniques,instruction sequences, and/or computer program products that embodytechniques of the present disclosure. However, it is understood that thedescribed disclosure may be practiced without these specific details.

In the present disclosure, the methods disclosed may be implemented assets of instructions or software readable by a device. Further, it isunderstood that the specific order or hierarchy of steps in the methodsdisclosed are instances of example approaches. Based upon designpreferences, it is understood that the specific order or hierarchy ofsteps in the method can be rearranged while remaining within thedisclosed subject matter. The accompanying method claims presentelements of the various steps in a sample order, and are not necessarilymeant to be limited to the specific order or hierarchy presented.

The described disclosure may be provided as a computer program product,or software, that may include a machine-readable medium having storedthereon instructions, which may be used to program a computer system (orother electronic devices) to perform a process according to the presentdisclosure. A machine-readable medium includes any tangible mechanismfor storing information in a form (e.g., software, processingapplication) readable by a machine (e.g., a computer). Themachine-readable medium may include, but is not limited to, magneticstorage medium (e.g., floppy diskette), optical storage medium (e.g.,CD-ROM); magneto-optical storage medium, read only memory (ROM); randomaccess memory (RAM); erasable programmable memory (e.g., EPROM andEEPROM); flash memory; or other types of medium suitable for storingelectronic instructions.

It is believed that the present disclosure and many of its attendantadvantages will be understood by the foregoing description, and it willbe apparent that various changes may be made in the form, constructionand arrangement of the components without departing from the disclosedsubject matter or without sacrificing all of its material advantages.The form described is merely explanatory, and it is the intention of thefollowing claims to encompass and include such changes.

While the present disclosure has been described with reference tovarious embodiments, it will be understood that these embodiments areillustrative and that the scope of the disclosure is not limited tothem. Many variations, modifications, additions, and improvements arepossible. More generally, embodiments in accordance with the presentdisclosure have been described in the context of particularimplementations. Functionality may be separated or combined in blocksdifferently in various embodiments of the disclosure or described withdifferent terminology. These and other variations, modifications,additions, and improvements may fall within the scope of the disclosureas defined in the claims that follow. While a number of examples ofaspects and embodiments have been discussed above, those of skill in theart will recognize certain modifications, permutations, additions andsub combinations thereof. It is therefore intended that the followingappended claims and claims hereafter introduced are interpreted toinclude all such modifications, permutations, additions andsub-combinations as are within their true spirit and scope.

1. A method of controlling building equipment comprising: receiving an image from an image sensor placed in a space associated with the building, the image including information representative of one or more occupants in the space; at a first processor coupled with a memory storing the image, determining, from the image, a presence of one or more occupants within the space; and providing a control signal to one or more controllable building elements based on the presence of the occupant within the space.
 2. The method of claim 1 further comprising: at the first processor coupled with the memory storing the image, determining a position of the one or more occupants in the space.
 3. The method of claim 2 further comprising: from a second processor in communication with the first processor, providing the control signal to the one or more controllable building elements based on the position of the occupant in the space.
 4. The method of claim 3 further comprising: providing the control signal to the one or more controllable building elements based on at least one of the presence of the one or more occupants in the space or the position of the one or more occupants in the space, and at least one luminance value associated with the space.
 5. The method of claim 3 wherein the controllable building elements include one or more of a dimmable lighting element, a variable ventilation element, a heating element or a cooling element; the method further comprising one or more of: providing a first control signal to the dimmable lighting element to illuminate the lighting element based on the position of the occupant in the space; providing a second control signal to the variable ventilation element to provide ventilation to the space based on position of the occupant in the space; providing a third control signal to the heating element to provide heating to the space based on the position of occupant in the space; or providing a fourth control signal to the cooling element to provide cooling to the space based on the position of the occupant in the space.
 6. The method of claim 1 further comprising: in the memory, storing digital image data from the space associated with the building, the digital image data including information representative of one or more occupants in the space; at the first processor coupled with the memory storing the digital image data, processing the digital image data using a Canny edge detection method, the processed digital image data providing image data representative of the one or more occupants in the space; and at the first processor, comparing the processed digital image data to at least one reference data set to identify the location of the one or more occupants in the space.
 7. The method of claim 6 further comprising: associating the one or more controllable building elements with one or more cells, and providing the control signal to the one or more controllable building elements based on the occupant value for the one or more cells associated with the one or more controllable building elements.
 8. The method of claim 6 further comprising, at the processor coupled with the memory storing the digital image data: applying a smoothing function to the occupant values for each cell to provide smoothed occupancy values for each cell; applying an offset to each smoothed occupancy value to provide offset occupancy values for each cell; normalizing the offset occupancy values on a region basis to provide normalized values for each cell; determining a mean value for each region to yield occupancy control values; and providing the control signal to the one or more controllable building elements in each region based on the yielded occupancy control values.
 9. The method of claim 1 further comprising: associating the space with a matrix including a dimensional representation of the space, the matrix including one or more regions and one more or more cells associated with each region; and assigning an occupant value for each cell based on the position of the occupants within the space.
 10. The method of claim 1 further comprising: at the first processor coupled with the memory storing the image, determining a position of the one or more occupants in the space; at the first processor coupled with the memory storing the image, calculating a centroid of occupancy of the one or more occupants in the space based on the position data; and providing the control signal to the one or more controllable building elements based on the calculated centroid of occupancy.
 11. A building management system comprising: an image processing occupancy sensor comprising: at least one image sensor; and at least one first microprocessor coupled with at least one memory, the microprocessor and memory configured to receive and store an image from the image sensor, the at least one memory including computer executable instructions comprising: a first set of instructions to identify an occupant within the image; a second set of instructions to determine the position of an occupant in a space associated with the image processing occupancy sensor and provide data indicative of the position of the occupant in the space.
 12. The building management system of claim 11 further comprising: at least one second microprocessor in communication with the at least one first microprocessor, the at least one second microprocessor coupled with at least one second memory, the at least one second microprocessor and second memory including computer executable instructions comprising a third set of instructions to control one more controllable elements in the space based on the position of the occupant in the space.
 13. The building management system of claim 12 wherein: the controllable building elements include one or more of a dimmable lighting elements, a variable ventilation element, a heating element or a cooling element; the at least one second microprocessor coupled with the at least one second memory including one or more computer executable instructions comprising: at least one fourth set of instructions to provide a first control signal to the dimmable lighting element to illuminate the lighting element based on the position of the occupant in the space; at least one fifth set of instructions to provide a second control signal to the variable ventilation element to provide ventilation to the space based on position of the occupant in the space; at least one sixth set of instructions to provide a third control signal to the heating element to provide heating to the space based on the position of occupant in the space; or at least one seventh set of instructions to provide a fourth control signal to the cooling element to provide cooling to the space based on the position of the occupant in the space.
 14. The building management system of claim 12, the third set of instructions to provide the control signal to the one or more controllable building elements based on at least one of the presence of the one or more occupants in the space or the position of the one or more occupants in the space, and at least one luminance value associated with the space.
 15. The building management system of claim 12, the at least one second microprocessor and second memory including computer executable instructions comprising: a fourth set of instructions to associate the space with a matrix including a dimensional representation of the space, the matrix including one or more regions and one more or more cells associated with each region; and a fifth set of instructions to assign an occupant value for each cell based on the position of the occupants within the space.
 16. The building management system of claim 15, the at least one second microprocessor and second memory including computer executable instructions comprising: a sixth set of instructions to associate the one or more controllable building elements with one or more cells; the third set of instructions to provide the control signal to the one or more controllable building elements based on the occupant value for the one or more cells associated with the one or more controllable building elements.
 17. An image processing occupancy sensor comprising: an image sensor; a microprocessor coupled with a memory, the memory configured to receive an image of a space from the image sensor, the memory including computer executable instructions to: determine a position of one or more occupants in the space and provide an output signal indicative of the position of the one more occupants in the space.
 18. The image processing occupancy sensor of claim 17, the computer executable instructions to determine a position of the one or more occupants in the space further configured to: process the image data using a Canny edge detection method, the processed image data providing image data representative of the one or more occupants in the space; and compare the processed digital image data to at least one reference data set to identify the location of the one or more occupants in the space.
 19. The image processing occupancy sensor of claim 18, the computer executable instructions further configured to calculate a centroid of occupancy of the one or more occupants in the space based on the position data. 